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(57) Abstract 

The invention provides a method and system for monitoring status in a relatively continuous, consistent, and intelligent manner. 
A status monitor receives monitoring data, adaptively and dynamically builds a database of known combinations of monitoring data, and 
adaptively and dynamically associates those known combinations with assessments of the monitored devices, systems, or networks. From an 
initial set of selected knowledge that is limited (even limited to no knowledge at all), the status monitor leams those anomalous conditions 
that require response and what responses are appropriate. The status monitor develops a database of information regarding distinguishable 
conditions, and measurements of the likely causes or effects of recognizable errors or faults. When an anomalous pattern is recognized, the 
status monitor, responsive to the anomalous pattern, diagnoses and corrects, or informs a human operator regarding, the monitored devices, 
systems, or network. The monitoring data includes a set of data streams each possibly having a different format, and each selectively 
interpreted so as to present information to the status monitor in a format usable by the status monitor. New data streams and formats can 
be dynamically added or altered. Appropriate responses can include informing human beings; taking remedial action for the monitored 
devices, systems, or networks; or altering or terminating the operation of the monitored devices, systems, or networks. 
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ADAPTIVE AND GENERALIZED STATUS MONITOR 



Background of the Invention 

5 L Field of the Invention 

The invention relates to status monitors, including those for adaptively 
monitoring status information from multiple sources such as file servers and system 
administrators. 

10 

2. Related Art 

Monitoring devices collect and present monitoring information, such as 
information regarding operation of a device, system, or network. The monitoring 
15 information is sometimes used to determine or respond to faults in the monitored 
devices, systems, or networks. 

One problem in the known art is that of recognizing and responding to 
anomalous behavior on the part of the monitored devices, systems, or networks. 
20 Because the monitored devices, systems, or networks can be complex, it is difficult or 
impossible to anticipate all, or even most, of the possible ways in which anomalous 
behavior can occur. Even if it were possible to anticipate anomalous behaviors, it is 
difficult or impossible to anticipate how those anomalous behaviors would manifest 
themselves in the avail-able data. 

25 

A first known method is to present a visual display of status 
information, and to rely on a human operator to determine whether the behavior of 
the monitored devices, systems, or networks are anomalous, and if so, to determine 
what that anomalous behavior indicates about possible errors or faults in operation. 
30 While this method can achieve the purpose of recognizing and responding to 
anomalous behaviors, it has the drawback of requiring constant and consistent 

1 



wo 00/68795 PCT/USOO/12491 

attention of a human being relatively skilled in the operation of the monitored 
devices, systems, or networks. This drawback is exacerbated when there are a 
relatively large number of monitored devices, systems, or networks or when the 
monitored devices, systems or networks are complex. Moreover, this known method 
5 is also subject to the drawback that it is limited to those aspects of behavior that are 
predetermined for presentation to the human being. 

A second known method is to present information, as in the first 
method, to an expert system or other software designed for recognizing and 

10 responding to anomalous behavior. While this known method has the advantage of 
not requiring the constant and consistent attention of a human being, it suffers from 
the drawback that it is limited by the skill predetermined for inclusion in the expert 
system or other software. As with the first known method, this method is also subject 
to the drawback that it is limited to those aspects of behavior that are predetermined 

15 for presentation (to the expert system). Moreover, this method is also subject to the 
drawback that it can erroneously determine and respond to anomalous conditions that 
are not in fact faults, without substantial opportunity to leam. 



Accordingly, it would be desirable to provide a method and system for 
20 monitoring status in a relatively continuous, consistent, and intelligent manner. This 
method is achieved in an embodiment of the invention in which a status monitor 
receives monitoring data, adaptively and dynamically builds a database of known 
combinations of monitoring data, and adaptively and dynamically associates those 
known combinations with assessments of the monitored devices, systems, or 
25 networks. From an initial set of selected knowledge that is limited (even limited to 
no knowledge at all), the status monitor can leam those anomalous conditions that 
require response and what responses are appropriate. 



2 
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Summary of the Invention 



The invention provides a method and system for monitoring status in a 
relatively continuous, consistent, and intelligent manner. A status monitor receives 
5 monitoring data, adaptively and dynamically builds a database of known 
combinations of monitoring data, and adaptively and dynamically associates those 
known combinations with assessments of the monitored devices, systems, or 
networks. From an initial set of selected knowledge that is limited (even limited to 
no knowledge at all), the status monitor learns those anomalous conditions that 

10 require response and what responses are appropriate. The status monitor develops a 
database of information regarding distinguishable conditions, and measurements of 
the likely causes or effects of recognizable errors or faults. When an anomalous 
pattern is recognized, the status monitor, responsive to the anomalous pattem, 
diagnoses and corrects, or informs a human operator regarding, the monitored 

15 devices, systems, or network. 

In a preferred embodiment, the monitoring data mcludes a set of data 
streams each possibly having a different format, and each selectively interpreted so as 
to present information to the status monitor in a format usable by the status monitor. 
20 New data streams and formats can be dynamically added or altered. Appropriate 
responses can include informing human beings; taking remedial action for the 
monitored devices, systems, or networks; or altering or terminating the operation of 
the monitored devices, systems, or networks. 

25 

Brief Description of the Drawings 

Figure 1 shows a block diagram of a system including a status monitor 
for adaptively monitoring status information from multiple sources. 

30 
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Figure 2 shows a block diagram of a status monitor for adaptively 
monitoring status information from multiple sources. 



Figure 3 shows a process flow diagram of a method of operation for a 
5 status monitor for adaptively monitoring status information from multiple sources. 

Detailed Description of the Preferred Embodiment 

In the following description, a preferred embodiment of the invention is 
10 described with regard to preferred process steps and data structures. However, those 
skilled in the art would recognize, after perusal of this application, that embodiments 
of the invention may be implemented using one or more general purpose processors 
(or special purpose processors adapted to the particular process steps and data 
structures) operating under program control, and that implementation of the preferred 
15 process steps and data structures described herein using such equipment would not 
require undue experimentation or further invention. 

System Elements 

20 Figure 1 shows a block diagram of a system including a status monitor 

for adaptively monitoring status information from multiple sources. 

A system 100 includes a set of data sources 110, a set of corresponding 
data interfaces 120, a status monitor 130, and a status recipient 140. 

25 

The data sources 1 10 can include differing types of data sources 1 10 for 
which monitoring is appropriate, including a file server 1 1 1 or other type of server, a 
system administrator 112 or other operator, an HVAV controller 113, a refinery; 
controller 114, a software element in a computer system 115 or other diagnostic 
30 sources 116. The differing types of data sources 110 can generate data in differing 
formats. For example, the file server 1 1 1 or other type of server can generate data in 

4 
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SNMP format; the system administrator 1 12 or other operator can generate data using 
an email program, and the other diagnostic sources 113 can generate data in other 
formats. SNMP format and email formats are known in the art of network 
communication. 

5 

In a preferred embodiment, the data sources 1 10 use a communication 
network to send information to the data interfaces 120. The communication network 
can include any known apparatus and methods for sending information from the data 
sources 1 10 to the data interfaces 120. Those skilled in the art would recognize, after 
10 perusal of this application, that any such apparatus and methods would be within the 
scope and spirit of the invention. In a preferred embodiment, the communication 
network includes a LAN (local area network), WAN (wide area network), an internet, 
intranet, extranet, VPN (virtual private network), or some combination thereof. 

15 The data interfaces 120 each correspond to one of the data sources 1 10. 

Each data interface 120 receives data from its corresponding data source 110, and 
forwards that data to the status monitor 130 in a format usable by the status monitor 
130. Additional data interfaces can be added, if desired. In a preferred embodiment, 
each data interface 120 is disposed for recognizing and parsing the format of its 

20 corresponding data source 1 10, and for generating messages in a single format usable 
by the status monitor 130. Moreover, data interfaces may completely encapsulate all 
knowledge of the format and the language of the data source. 

The status monitor 130 includes a processor, program and data 
25 memory, and can include mass storage. Construction and use of devices including 
processors, program and data memory, and mass storage are known in the art of 
computer programming. 

The status monitor 130 need not be a separate physical device. It can 
30 be embodied in a software element in a device also used for other purposes, and can 
be physically co-located with the status recipient 140. In a preferred embodiment, 

5 
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software elements of the status monitor 130 operate as an application program under 
control of an operating system on the processor with program and data memory. The 
application program can include software derived from source code compiled or 
interpreted from a Perl script or one or more programming languages such as the C-H- 
5 programming language. Both the Perl scripting language and the C++ programming 
language are known in the art of computer programming. 

The status monitor 130 receives messages from the data interfaces 120, 
and is disposed for processing those messages to recognize fault conditions and to 
10 determine the nature of the fault with which the fault conditions are correlated. 

As used herein, the term "fault" and the phrase "fault condition" refer to 
conditions of interest to operators of the system 110, such as human operators or 
control programs. There is no particular requirement in the invention that a fault or 
15 fault condition refer to an actual error or failure in operation of the system 100 or one 
of its parts. 

When recognizing fault conditions and determining the nature of the 
correlated faults, the status monitor 130 sends a message to the status recipient 140 
20 indicating the fault conditions and the faults. 

The status recipient 140 can include an operator of the system 1 10, such 
as a human operator or a control program, a log file, or a communication link for 
distributing messages regarding the fault conditions and the faults. 

25 

The status recipient 140 can include a workstation for use by the 
operator of the system 110, logically remote from the device 111, which can be 
physically relatively local or physically relatively remote. In a preferred 
embodiment, the system can include more than one such device 1 1 1 being monitored, 
30 and more than one such status recipient 140 disposed for receiving monitoring 
information. 

6 
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The workstation for the status recipient can include a monitoring and 
analysis program, including a graphical user interface and a set of commands for 
analyzing and presenting data. 

5 

Status Monitor Elements 

Figure 2 shows a block diagram of a status monitor for adaptively 
10 monitoring status information from multiple sources. 

The status monitor 130 includes multiple data input ports 210, each of 
which is associated with a corresponding data interface 120 and a corresponding filter 
211. Each input port 210 receives messages indicating values for raw data 201 from 
15 its corresponding data interface 120. Each input port 210 processes the raw data 201 
to provide regularized data 202, and sends the regularized data 202 to a 
corresponding comparison element 220. 

As used herein, the phrase "regularized data" 202 refers only to a form 
20 of the raw data 201 after the input port 210 has processed it. There is no particular 
requirement that the regularized data 202 must follow some known distribution, 
although it is expected that many items of raw data 201 will have known random 
distributions such as a normal, binomial, poisson, or equiprobable distributions. 

25 In a preferred embodiment, the input ports 210 may regularize the raw 

data 201 by determining a trend. The input ports 210 can determine a trend using any 
one of a number of known techniques, including for example relative time change in 
the raw data 201. In alternative embodiments, the input ports 210 can regularize the 
raw data 201 by determining other statistical measures, such as confidence values or 

30 correlation values. 



7 
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The comparison elements 220 each receive the regularized data 202, 
and determine if the received values for the regularized data 202 are outside of a 
selected limit range, designated by a selected lower limit value 203 and a selected 
upper limit value 204. Each comparison elements 220 provides a corresponding out- 
5 of-limit indicator bit 205 indicating whether or not the regularized data 202 is within 
the selected limit range. 

The indicator bits 205 from the comparison elements 220 are collected 
into an indicator bh vector 206. The indicator bit vector 206 is coupled to a bit vector 
10 comparator 230. 

The bit vector comparator 230 includes a bit vector memory 23 1, which 
itself includes a set of selected bit vectors 206, each associated with a fault descriptor 
207. The fault descriptor 207 indicates information about a fault associated with its 
1 5 corresponding bit vector 206 . 

In a preferred embodiment, the number of bit vectors 206 in the bit 
vector memory 231 can be selected by a system administrator 1 12 or other operator, 
and is preferably at least about 32. 
20 In a preferred embodiment, the fault descriptor 207 includes a pointer to 

a data structure 208 that includes further information about the fault. This further 
information can include one or more of, or some combination of, the following: 

o an assessment of the fault, such as a numeric degree of seriousness; 

25 

o a description of the fault, such as a title or text description; 
or 

o a set of actions to be taken in response to the fault, such as a set of individuals 

30 to inform about the fault (whether by email, pager, or other technique), a set of 



8 
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functions for the system 100 that should be suspended in response to the fault, 
or other appropriate actions. 

The bit vector comparator 230 receives the indicator bit vector 206 and 
5 compares it against the selected bit vectors 206 in the bit vector memory 23 1 . The bit 
vector comparator 230 selects one or more matching selected bit vectors 206 and 
provides, in response to associated fault descriptors 207, one or more outputs. 

In a preferred embodiment, the bit vector comparator 230 selects the 
10 "best match" among the selected bit vectors 206 in the bit vector memory 23 1 for the 
indicator bit vector 206, and provides one output in response to the corresponding 
fauh descriptor 207. The bit vector comparator 230 sends the indicator bit vector 206 
and the corresponding fault descriptor 207 to the status recipient 140, and takes other 
appropriate action as indicated by the fault descriptor 207. 

15 

In a preferred embodiment, at least one (and possibly several) of the 
selected bit vectors 206 in the bit vector memory 231 has an associated fauh 
descriptor 207 that describes a "normal" or non-fault condition. Thus, the bit vector 
comparator 230 can select, in response to the input bit vector 206, an associated 
20 "normal" fault descriptor 207. Thus, some anomalous bit vectors 206 can be 
associated with known lack of error. 

In a preferred embodiment, the "normal" fault descriptor 207 can be 
selected to indicate that all is well with the system 100 and that no action is required. 
25 Moreover, the "normal" fault descriptor 207 (and other fault descriptor 207 deemed 
insufficiently serious) can be set so that no action is taken in response thereto, 
including sending no message to the status recipient 140. 

In a preferred embodiment, the selected techniques or values used by 
30 the system 100 can be included in a configuration database 132 associated with the 
status monitor 130 and alterable by the system administrator 112 or other operator. 

9 
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The configuration database 132 can include one or more of, or any combination of, 
any of the following: 



o The technique(s) used by each data interface 120 to reformat the data from the 
5 data sources 110. For example, data interfaces 120 disposed for receiving 

SNMP messages can be configured to recognize and extract data from those 
messages. Data interfaces 120 disposed for receiving email or other text can 
be configured to recognize text in response to selected keywords and to asses 
that text in response thereto. 

10 

o The technique(s) used by each input port 210 to determine trends. 

o Known associations between selected bit vector patterns and selected faults or 
other events. 

15 

In a preferred embodiment, the configuration database 132 can include 
a set of possible anomalies that might be associated with the functional status of the 
device 1 1 1 and an set of associations between those anomalies and a set of selected 
fault conditions. 

20 

Method of Operation 

Figure 3 shows a process flow diagram of a method of operation for a 
status monitor for adaptively monitoring status information from multiple sources. 

25 

A method 300 is performed by the system 100 operating in conjunction, 
including the data sources 1 10, data interfaces 120, and the status monitor 130. 

At a flow point 310, the system 100 is in operation and the method 300 
30 is being continuously performed. 

10 
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At a step 31 1, the data sources 1 10 provide data to the data interfaces 
120. In a preferred embodiment, the data sources 110 provide data by sending 
messages to the data interfaces 120 in known formats, as described above. 

At a step 312, each data interface 120 can receive data from its 
corresponding data source 110. For each data interface 120 that receives data, the 
data interface 120 (a) receives the data, (b) reformats the data if necessary into a the 
format usable by the status monitor 130, and (c) sends the reformatted data to the 
status monitor 130 in that usable format. 

At a step 313, the input ports 210 of the status monitor 130 can each 
receive a set of values for raw data 201. For each input port 210 that receives raw 
data 201, the input port 210 (a) receives the raw data 201, (b) processes the raw data 
201 to provide regularized data 202, and (c) sends the regularized data 202 to its 
corresponding comparison element 220 in the status monitor 130. 

As part of this step, each input port 210 that receives raw data 201 can 
determine a trend for that raw data 20 1, as described above. 

20 At a step 314, the comparison elements 220 in the status monitor 130 

can each receive a set of values for the regularized data 202. For each comparison 
element 220 that receives regularized data 202, the comparison element 220 (a) 
receives the regularized data 202, and (b) processes the regularized data 202 to 
determine if the received values for the regularized data 202 are outside of a selected 

25 limit range, as described above. In response to this processing, the comparison 
element 220 provides a corresponding out-of-limit indicator bit 205 indicating 
whether or not the regularized data 202 is within the selected limit range. 

At a step 315, the indicator bits 205 from the comparison elements 220 
30 are collected into an indicator bit vector 206. The indicator bit vector 206 is coupled 
to a bit vector comparator 230. 

11 
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At a step 316, the bit vector comparator 230 receives the indicator bit 
vector 206 and compares it against the selected bit vectors 206 in the bit vector 
memory 231. 

5 

At a step 3 17, in response to the comparison in the previous step, the bit 
vector comparator 230 selects one or more matching selected bit vectors 206 and 
provides, in response to fauU descriptors 207 associated with those matching selected 
bit vectors 206, one or more outputs. In a preferred embodiment, the bit vector 
10 comparator 230 selects one "best match" among the selected bit vectors 206 in the bit 
vector memory 231 for the indicator bit vector 206, and provides one output in 
response to the corresponding fault descriptor 207, as described above. 

At a step 3 18, in response to the fault descriptors 207 determined in the 
15 previous step, the bit vector comparator 230 sends the indicator bit vector 206 and the 
corresponding fault descriptor 207 to the status recipient 140, and takes other 
appropriate action as indicated by the fault descriptor 207. 

The method 300 operates continuously, and so returns to the flow point 

20 310. 

In a preferred embodiment, the system 100 starts with substantially no 
information in the bit vector memory 231, and so spends an amount of time in a 
learning phase. During the learning phase, the status monhor 130 determines that 
25 indicator bit vectors 206 that do not well match any of the selected bit vectors 206 in 
the bit vector memory 23 1 are new bit vectors 206, and adds those new bit vectors 
206 to the bit vector memory 23 1 . 

When recognizing a new bit vector 206, the status monitor 130 can send 
30 a message to the status recipient 140 requesting information to associate in the fault 
descriptor 207 for that new bit vector 206. 

12 
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When recognizing a new bit vector 206, the status monitor 130 can also 
adaptively respond to other information available at the time the new bit vector 206 is 
received, including one or more of, or any combination of, any of the following: 

o Selected patterns can be associated with keywords or other aspects (such as 
priority) of email received from selected users. 

o Selected anomalous patterns can be associated with normal activity. For 
example, period of low network activity in the absence of other factors may be 
associated with off-peak hours. 

o Selected anomalous patterns can be associated with specific defects based 
upon past history. 

or 

o Selected anomalous pattems can be associated with preset data that is included 
in the configuration database 132. 

Alternative Embodiments 

Although preferred embodiments are disclosed herein, many variations 
are possible which remain within the concept, scope, and spirit of the invention, and 
these variations would become clear to those skilled in the art after perusal of this 
application. 



13 
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1 . A method including steps for 
receiving monitoring data; 

5 in response to said monitoring data, adaptively and dynamically 

building a database of known combinations of said monitoring data; 

in response to said monitoring data, adaptively and dynamically 
building a database of associations between said knovra combinations and selected 
monitoring assessments; 
10 taking action in response to said selected monitoring assessments. 

2. A method as in claim 1, wherein said database includes a non- 
null initial set of said known combinations. 

15 3, A method as in claim 1, wherein said database includes a null 

initial set of said known combinations. 

4. A method as in claim 1, wherein said database includes at least 
one said known combination associated with a non-fault monitoring assessment. 

20 

5. A method as in claim 1, wherein said monitoring data includes 
SNMP data or email text. 

6. A method as in claim 1, wherein said selected monitoring 
25 assessments include at least one of 

diagnostic information of a fault; 

message information for presentation to an operator; or 

remedial action to be taken in response to said selected monitoring 

assessment. 

30 

14 
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7. A method as in claim 1, wherein said steps for receiving 
monitoring data include steps for receiving selected monitoring data having a new 
data format. 



8. Apparatus including 

a plurality of monitoring data input ports, each providing a sequence of 
monitoring data; 

a plurality of comparison elements, each one coupled to an associated 
one of said input ports, and each providing a sequence of comparison values; 

a database having a plurality of sets of comparison values, each said set 
of comparison values having a monitoring assessment associated therewith; 

a vector comparator, coupled to said comparison values for said 
comparison elements, and coupled to said database, and providing a selected group of 
said plurality of sets in response thereto; and 

a fault response element, coupled to at least one said monitoring 
assessment, and providing a response thereto. 

9. Apparatus as in claim 8, wherein at least one said sequence of 
monitoring data includes SNMP data or email text. 

10. Apparatus as in claim 8, wherein said data input ports each 
include an element for recognizing monitoring data having a selected data format, 
whereby said apparatus can receive a new said data input port having a new said data 
format. 

IL Apparatus as in claim 8, wherein said database includes a non- 
null initial set of said comparison values, 

12. Apparatus as in claim 8, wherein said database includes a null 
initial set of said comparison values. 



15 
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13. Apparatus as in claim 8, wherein said database includes at least 
one said known set of comparison values associated with a non-fault monitoring 
assessment. 



14, Apparatus as in claim 8, wherein said selected monitoring 
assessments include at least one of 

diagnostic information of a fault; 

message information for presentation to an operator; or 

remedial action to be taken in response to said selected monitoring 

assessment. 



15. Apparatus including memory storing a computer program, said 
computer program including 

a database having a plurality of sets of comparison values, each said set 
of comparison values having a monitoring assessment associated therewith; and 

a vector comparator, coupled to said comparison values for said 
comparison elements, and coupled to said database, and providing a selected group of 
said plurality of sets in response thereto. 

16. A program as in claim 15, wherein said database includes a non- 
null initial set of said comparison values. 



17. A program as in claim 15, wherein said database includes a null 
initial set of said comparison values. 

18. A program as in claim 15, wherein said database includes at least 
one said comparison values associated with a non-fault monitoring assessment. 

19. A program as in claim 15, wherein said selected monitoring 
assessments include at least one of 

diagnostic information of a fault; 

16 
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message information for presentation to an operator; or 

remedial action to be taken in response to said selected monitoring 

assessment. 

20. Apparatus including memory storing a data structure, said data 
structure including a plurality of sets of comparison values, each said set of 
comparison values having a monitoring assessment associated therewith. 



21. A structure as in claim 20, including at least one said known 
10 combination associated with a non-fault monitoring assessment. 

22. In a monitoring apparatus, a computer program including 

a database having a plurality of sets of comparison values, each said set 
of comparison values having a monitoring assessment associated therewith; and 
15 a vector comparator, coupled to said comparison values for said 

comparison elements, and coupled to said database, and providing a selected group of 
said plurality of sets in response thereto. 



23. A program as in claim 22, wherein said database includes a non- 
20 null initial set of said comparison values, 

24. A program as in claim 22, wherein said database includes a null 
initial set of said comparison values. 

25 25. A program as in claim 22, wherein said database includes at least 

one said comparison values associated with a non- fault monitoring assessment. 

26. A program as in claim 22, wherein said selected monitoring 
assessments include at least one of 
30 diagnostic information of a fault; 

message information for presentation to an operator; or 

17 
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remedial action to be taken in response to said selected monitoring 

assessment. 



27. In a monitoring method, a data structure including a plurality of 
5 sets of comparison values, each said set of comparison values having a monitoring 

assessment associated therewith. 

28. A structure as in claim 27, including at least one said known 
combination associated with a non-fault monitoring assessment. 
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310 

System 100 is in operation. 
Metiiod 300 is being performed. 



311 

Data sources 110 provide data to 
data interfaces. 



▼ 

312 

Data interface 120 receives data, 
reformats it (if necessary), and 
sends data to monitor in a usable 
format. 

i 

313 

Input ports 210 receive raw data 

201, process raw data 201 to 
provide regularized data 202, and 
send regularized data 202 to 
comparison element 220. 



314 

Comparison element 220 
determines if regularized data 
202 is outside a selected range 
limit and provides an out-of-limit 
indicator bit. 



Figure 3 




317 



Bit vector comparator 230 
selects matching bit vector 

206 and provides an output in 
response to the fault 

descriptor 207. 

i 

318 

Bit vector comparator 230 
sends indicator bit vector 206 

and corresponding fault 
descriptor to status recipient 
140 

319 

Method 300 retums to a flow 

point 3 1 0 for continuous 
operation. 



t 

315 

Indicator bits 205 are collected 
into indicator bit vector 206, 



316 

Bit vector comparator 230 
receives indicator bit vector 206 

and compares it to selected 
values in bit vector memory 23 1 . 
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